Implementation of large, multi-site hospital interventions: a realist evaluation of strategies for developing capability.

BACKGROUND: This study presents guidelines for implementation distilled from the findings of a realist evaluation. The setting was local health districts in New South Wales, Australia that implemented three clinical improvement initiatives as part of a state-wide program. We focussed on implementation strategies designed to develop health professionals' capability to deliver value-based care initiatives for multisite programs. Capability, which increases implementers' ability to cope with unexpected scenarios is key to managing change. METHODS: We used a mixed methods realist evaluation which tested and refined program theories elucidating the complex dynamic between context (C), mechanism (M) and outcome (O) to determine what works, for whom, under what circumstances. Data was drawn from program documents, a realist synthesis, informal discussions with implementation designers, and interviews with 10 key informants (out of 37 identified) from seven sites. Data analysis employed a retroductive approach to interrogate the causal factors identified as contributors to outcomes. RESULTS: CMO statements were refined for four initial program theories: Making it Relevant- where participation in activities was increased when targeted to the needs of the staff; Investment in Quality Improvement- where engagement in capability development was enhanced when it was valued by all levels of the organisation; Turnover and Capability Loss- where the effects of staff turnover were mitigated; and Community-Wide Priority- where there was a strategy of spanning sites. From these data five guiding principles for implementers were distilled: (1) Involve all levels of the health system to effectively implement large-scale capability development, (2) Design capability development activities in a way that supports a learning culture, (3) Plan capability development activities with staff turnover in mind, (4) Increased capability should be distributed across teams to avoid bottlenecks in workflows and the risk of losing key staff, (5) Foster cross-site collaboration to focus effort, reduce variation in practice and promote greater cohesion in patient care. CONCLUSIONS: A key implementation strategy for interventions to standardise high quality practice is development of clinical capability. We illustrate how leadership support, attention to staff turnover patterns, and making activities relevant to current issues, can lead to an emergent learning culture.

Impact of mental disorders on unplanned readmissions for congestive heart failure patients: a population-level study.

AIMS: Reducing preventable hospitalization for congestive heart failure (CHF) patients is a challenge for health systems worldwide. CHF patients who also have a recent or ongoing mental disorder may have worse health outcomes compared with CHF patients with no mental disorders. This study examined the impact of mental disorders on 28 day unplanned readmissions of CHF patients. METHODS AND RESULTS: This retrospective cohort study used population-level linked public and private hospitalization and death data of adults aged ≥18 years who had a CHF admission in New South Wales, Australia, between 1 January 2014 and 31 December 2020. Individuals' mental disorder diagnosis and Charlson comorbidity and hospital frailty index scores were derived from admission records. Competing risk and cause-specific risk analyses were conducted to examine the impact of having a mental disorder diagnosis on all-cause hospital readmission. Of the 65 861 adults with index CHF admission discharged alive (mean age: 78.6 ± 12.1; 48% female), 19.2% (12 675) had at least one unplanned readmission within 28 days following discharge. Adults with CHF with a mental disorder diagnosis within 12 months had a higher risk of 28 day all-cause unplanned readmission [hazard ratio (HR): 1.21, 95% confidence interval (CI): 1.15-1.27, P-value < 0.001], particularly those with anxiety disorder (HR: 1.49, 95% CI: 1.35-1.65, P-value < 0.001). CHF patients aged ≥85 years (HR: 1.19, 95% CI: 1.11-1.28), having ≥3 other comorbidities (HR: 1.35, 95% CI: 1.25-1.46), and having an intermediate (HR: 1.34, 95% CI: 1.28-1.40) or high (HR: 1.37, 95% CI: 1.27-1.47) frailty score on admission had a higher risk of unplanned readmission. CHF patients with a mental disorder who have ≥3 other comorbidities and an intermediate frailty score had the highest probability of unplanned readmission (29.84%, 95% CI: 24.68-35.73%) after considering other patient-level factors and competing events. CONCLUSIONS: CHF patients who had a mental disorder diagnosis in the past 12 months are more likely to be readmitted compared with those without a mental disorder diagnosis. CHF patients with frailty and a mental disorder have the highest probability of readmission. Addressing mental health care services in CHF patient's discharge plan could potentially assist reduce unplanned readmissions.

Four System Enablers of Large-System Transformation in Health Care: A Mixed Methods Realist Evaluation.

Policy Points The implementation of large-scale health care interventions relies on a shared vision, commitment to change, coordination across sites, and a spanning of siloed knowledge. Enablers of the system should include building an authorizing environment; providing relevant, meaningful, transparent, and timely data; designating and distributing leadership and decision making; and fostering the emergence of a learning culture. Attention to these four enablers can set up a positive feedback loop to foster positive change that can protect against the loss of key staff, the presence of lone disruptors, and the enervating effects of uncertainty. CONTEXT: Large-scale transformative initiatives have the potential to improve the quality, efficiency, and safety of health care. However, change is expensive, complex, and difficult to implement and sustain. This paper advances system enablers, which will help to guide large-scale transformation in health care systems. METHODS: A realist study of the implementation of a value-based health care program between 2017 and 2021 was undertaken in every public hospital (n = 221) in New South Wales (NSW), Australia. Four data sources were used to elucidate initial program theories beginning with a set of literature reviews, a program document review, and informal discussions with key stakeholders. Semistructured interviews were then conducted with 56 stakeholders to confirm, refute, or refine the theories. A retroductive analysis produced a series of context-mechanism-outcome (CMO) statements. Next, the CMOs were validated with three health care quality expert panels (n = 51). Synthesized data were interrogated to distill the overarching system enablers. FINDINGS: Forty-two CMO statements from the eight initial program theory areas were developed, refined, and validated. Four system enablers were identified: (1) build an authorizing environment; (2) provide relevant, authentic, timely, and meaningful data; (3) designate and distribute leadership and decision making; and (4) support the emergence of a learning culture. The system enablers provide a nuanced understanding of large-system transformation that illustrates when, for whom, and in what circumstances large-system transformation worked well or worked poorly. CONCLUSIONS: System enablers offer nuanced guidance for the implementation of large-scale health care interventions. The four enablers may be portable to similar contexts and provide the empirical basis for an implementation model of large-system value-based health care initiatives. With concerted application, these findings can pave the way not just for a better understanding of greater or lesser success in intervening in health care settings but ultimately to contribute higher quality, higher value, and safer care.

The role of frailty risk for fracture-related hospital readmission and mortality after a hip fracture.

BACKGROUND: Frailty risk estimated using hospital administrative data may provide a useful clinical tool to identify older hip fracture patients at-risk of fracture-related readmissions and mortality. This study examined hip fracture hospitalisation temporal trends and explore the role of frailty risk in fracture-related readmission and mortality. METHODS: This retrospective cohort study was conducted using linked hospital admission and mortality data in New South Wales, Australia. Patients aged ≥65 years were admitted after a hip fracture between 2014 and 2021 for temporal trends and those admitted and discharged after a hip fracture in 2014-2018 for fracture-related readmission. The Hospital Frailty Risk Score was estimated, and patients were followed for at least 36 months after discharge. A semi-competing risk analysis was used to examine the associations of frailty with fracture-related readmission and/or mortality. RESULTS: Hip fracture hospitalisation rate was 472 per 100,000 and declined by 2.9 % (95 % confidence intervals (CI): -3.7 to -2.1) annually. Amongst 28,567 patients, 9.8 % were identified with low frailty risk, 39.4 %, intermediate frailty risk, and 50.6 % with high frailty risk. Patients with intermediate or high frailty risk had a higher chance of fracture-related readmission (Hazard ratios (HR): 1.33, 95 %CI: 1.21-1.47, HR: 1.65, 95 %CI: 1.49-1.83), death (HR: 1.50, 95 %CI: 1.38-1.63, HR: 1.80, 95 %CI: 1.65-1.96) and death post fracture-related readmission (HR: 1.32, 95 %CI: 1.12-1.56, HR: 1.56, 95 %CI: 1.32-1.84) than those with low frailty risk. CONCLUSIONS: It appears that frailty risk estimated using hospital administrative data can contribute to identify patients who could benefit from targeted interventions to prevent further fractures.

Audit and feedback to reduce unwarranted clinical variation at scale: a realist study of implementation strategy mechanisms.

BACKGROUND: Unwarranted clinical variation in hospital care includes the underuse, overuse, or misuse of services. Audit and feedback is a common strategy to reduce unwarranted variation, but its effectiveness varies widely across contexts. We aimed to identify implementation strategies, mechanisms, and contextual circumstances contributing to the impact of audit and feedback on unwarranted clinical variation. METHODS: Realist study examining a state-wide value-based healthcare program implemented between 2017 and 2021 in New South Wales, Australia. Three initiatives within the program included audit and feedback to reduce unwarranted variation in inpatient care for different conditions. Multiple data sources were used to formulate the initial audit and feedback program theory: a systematic review, realist review, program document review, and informal discussions with key program stakeholders. Semi-structured interviews were then conducted with 56 participants to refute, refine, or confirm the initial program theories. Data were analysed retroductively using a context-mechanism-outcome framework for 11 transcripts which were coded into the audit and feedback program theory. The program theory was validated with three expert panels: senior health leaders (n = 19), Agency for Clinical Innovation (n = 11), and Ministry of Health (n = 21) staff. RESULTS: The program's audit and feedback implementation strategy operated through eight mechanistic processes. The strategy worked well when clinicians (1) felt ownership and buy-in, (2) could make sense of the information provided, (3) were motivated by social influence, and (4) accepted responsibility and accountability for proposed changes. The success of the strategy was constrained when the audit process led to (5) rationalising current practice instead of creating a learning opportunity, (6) perceptions of unfairness and concerns about data integrity, 7) development of improvement plans that were not followed, and (8) perceived intrusions on professional autonomy. CONCLUSIONS: Audit and feedback strategies may help reduce unwarranted clinical variation in care where there is engagement between auditors and local clinicians, meaningful audit indicators, clear improvement plans, and respect for clinical expertise. We contribute theoretical development for audit and feedback by proposing a Model for Audit and Feedback Implementation at Scale. Recommendations include limiting the number of audit indicators, involving clinical staff and local leaders in feedback, and providing opportunities for reflection.

Guiding principles for effective collaborative implementation strategies for multisite hospital improvement initiatives: a mixed-method realist evaluation of collaborative strategies used in four multisite initiatives at public hospitals in New South Wales, Australia.

OBJECTIVE: Large-scale, multisite hospital improvement initiatives can advance high-quality care for patients. Implementation support is key to adoption of change in this context. Strategies that foster collaboration within local teams, across sites and between initiative developers and users are important. However not all implementation strategies are successful in all settings, sometimes realising poor or unintended outcomes. Our objective here is to develop guiding principles for effective collaborative implementation strategies for multi-site hospital initiatives. DESIGN: Mixed-method realist evaluation. Realist studies aim to examine the underlying theories that explain differing outcomes, identifying mechanisms and contextual factors that may trigger them. SETTING: We report on collaborative strategies used in four multi-site initiatives conducted in all public hospitals in New South Wales, Australia (n>100). PARTICIPANTS: Using an iterative process, information was gathered on collaborative implementation strategies used, then initial programme theories hypothesised to underlie the strategies' outcomes were surfaced using a realist dialogic approach. A realist interview schedule was developed to elicit evidence for the posited initial programme theories. Fourteen participants from 20 key informants invited participated. Interviews were conducted via Zoom, transcribed and analysed. From these data, guiding principles of fostering collaboration were developed. RESULTS: Six guiding principles were distilled: (1) structure opportunities for collaboration across sites; (2) facilitate meetings to foster learning and problem-solving across sites; (3) broker useful long-term relationships; (4) enable support agencies to assist implementers by giving legitimacy to their efforts in the eyes of senior management; (5) consider investment in collaboration as effective well beyond the current projects; (6) promote a shared vision and build momentum for change by ensuring inclusive networks where everyone has a voice. CONCLUSION: Structuring and supporting collaboration in large-scale initiatives is a powerful implementation strategy if contexts described in the guiding principles are present.

Incorporating a polygenic risk score-triaged coronary calcium score into cardiovascular disease examinations to identify subclinical coronary artery disease (ESCALATE): Protocol for a prospective, nonrandomized implementation trial.

BACKGROUND: Identifying and targeting established modifiable risk factors has been a successful strategy for reducing the burden of coronary artery disease (CAD) at the population-level. However, up to 1-in-4 patients who present with ST elevation myocardial infarction do so in the absence of such risk factors. Polygenic risk scores (PRS) have demonstrated an ability to improve risk prediction models independent of traditional risk factors and self-reported family history, but a pathway for implementation has yet to be clearly identified. The aim of this study is to examine the utility of a CAD PRS to identify individuals with subclinical CAD via a novel clinical pathway, triaging low or intermediate absolute risk individuals for noninvasive coronary imaging, and examining the impact on shared treatment decisions and participant experience. TRIAL DESIGN: The ESCALATE study is a 12-month, prospective, multicenter implementation study incorporating PRS into otherwise standard primary care CVD risk assessments, to identify patients at increased lifetime CAD risk for noninvasive coronary imaging. One-thousand eligible participants aged 45 to 65 years old will enter the study, which applies PRS to those considered low or moderate 5-year absolute CVD risk and triages those with CAD PRS ≥80% for a coronary calcium scan. The primary outcome will be the identification of subclinical CAD, defined as a coronary artery calcium score (CACS) >0 Agatston units (AU). Multiple secondary outcomes will be assessed, including baseline CACS ≥100 AU or ≥75th age-/sex-matched percentile, the use and intensity of lipid- and blood pressure-lowering therapeutics, cholesterol and blood pressure levels, and health-related quality of life (HRQOL). CONCLUSION: This novel trial will generate evidence on the ability of a PRS-triaged CACS to identify subclinical CAD, as well as subsequent differences in traditional risk factor medical management, pharmacotherapy utilization, and participant experience. TRIAL REGISTRATION: Australian New Zealand Clinical Trials Registry, ACTRN12622000436774. Trial was prospectively registered on March 18, 2022. https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=383134.

Horizon scanning, rapid reviews and living evidence to support decision-making: lessons from the work of the Critical Intelligence Unit in New South Wales, Australia during the COVID-19 pandemic.

The COVID-19 pandemic has seen an increase in rapidly disseminated scientific evidence and highlighted that traditional evidence synthesis methods, such as time and resource intensive systematic reviews, may not be successful in responding to rapidly evolving policy and practice needs. In New South Wales (NSW) Australia, the Critical Intelligence Unit (CIU) was established early in the pandemic and acted as an intermediary organisation. It brought together clinical, analytical, research, organisational and policy experts to provide timely and considered advice to decision-makers. This paper provides an overview of the functions, challenges and future implications of the CIU, particularly the Evidence Integration Team. Outputs from the Evidence Integration Team included a daily evidence digest, rapid evidence checks and living evidence tables. These products have been widely disseminated and used to inform policy decisions in NSW, making valuable impacts. Changes and innovations to evidence generation, synthesis and dissemination in response to the COVID-19 pandemic provide an opportunity to shift the way evidence is used in future. The experience and methods of the CIU have potential to be adapted and applied to the broader health system nationally and internationally.

Falls from ladders in New South Wales: A data-linkage study.

INTRODUCTION: Fall from ladders is increasingly identified as a significant cause of injury and mortality, yet large-scale research into ladder fall outcomes and trends is limited. OBJECTIVES: To explore the nature and severity of injuries resulting from ladder falls and to determine predictors of Injury Severity Score (ISS) and 6-month mortality. METHODS: Data were obtained from the New South Wales (NSW) Trauma Registry, Admitted Patient Data Collection and Registry of Births, Deaths, and Marriages on patients aged 15 and over who had major trauma from a ladder fall and were admitted to hospital between January 1st, 2012, and July 31st, 2019. Data linkage and descriptive statistics were carried out alongside bivariate and multivariable regression analysis. RESULTS: 963 patients injured after ladder falls were identified. The mean age was 61.9 years (SD 14.2), 91.0% were male, and 489 (50.8%) were born in Australia. The height of fall was between one and five meters in 827 (86.2%) patients, and the place of fall was home and residential places in 27.5%. The most common body areas injured were the head (26.5%), spine (21.2%) and thorax (20.6%), and the median injury severity score was 17. The median length of stay of patients' in-hospital and intensive care unit was six days and two days, respectively. Six months post-discharge mortality was 6.4%. The unadjusted association between the presence of comorbidities or socio-economic class and ISS or mortality was not statistically significant. Increasing ISS was found to be associated with increasing age (Estimate (Est), 15.2; 95% Confidence Interval (CI), 12.3-18.1) and a fall height greater than five metres (Est, 5.8; CI, 3.2-8.4). Mortality was found to be associated with increasing age (Odds ratio (OR), 1.06; CI, 1.03-1.08) and increasing ISS (OR, 1.19; CI, 1.15-1.24). CONCLUSION: People presenting to the hospital after falling from a ladder were predominately male, aged over 60 and had fallen in a residential setting. Increasing age and fall height are associated with more severe injuries.

Determinants of long-term unplanned readmission and mortality following self-inflicted and non-self-inflicted major injury: a retrospective cohort study.

PURPOSE: To describe the characteristics of major injury and identify determinants of long-term unplanned readmission and mortality after self-inflicted and non-self-inflicted injury to inform potential readmission screening. METHOD: A retrospective cohort study of 11,269 individuals aged ≥ 15 years hospitalised for a major injury during 2013-2017 in New South Wales, Australia. Unplanned readmission and mortality up to 27-month post-injury were examined. Logistic regression was used to examine predictors of unplanned readmission. RESULTS: During the 27-month follow-up, 2700 (24.8%) individuals with non-self-inflicted and 98 (26.1%) with self-inflicted injuries had an unplanned readmission. Individuals with an anxiety-related disorder and a non-self-inflicted injury who were discharged home were three times more likely (OR: 3.27; 95%CI 2.28-4.69) or if they were discharged to a psychiatric facility were four times more likely (OR: 4.11; 95%CI 1.07-15.80) to be readmitted. Compared to individuals aged 15-24 years, individuals aged ≥ 65 years were 3 times more likely to be readmitted (OR 3.12; 95%CI 2.62-3.70). Individuals with one (OR 1.60; 95%CI 1.39-1.84) or ≥ 2 (OR 1.88; 95%CI 1.52-2.32) comorbidities, or who had a drug-related dependence (OR 1.88; 95%CI 1.52-2.31) were more likely to be readmitted. The post-discharge age-adjusted mortality rate following a self-inflicted injury (35.6%; 95%CI 29.9-41.8) was higher than for individuals with a non-self-inflicted injury (11.0%; 95%CI 10.4-11.8). CONCLUSIONS: Unplanned readmission after injury is associated with injury intent, age, and comorbid health. Screening for anxiety and drug-related dependence after major injury, accompanied by service referrals and post-discharge follow-up, has potential to prevent readmission.

Improving access to primary health care: a cross-case comparison based on an a priori program theory.

BACKGROUND: Inequitable access to primary health care (PHC) remains a problem for most western countries. Failure to scale up effective interventions has been due, in part, to a failure to share the logic and essential elements of successful programs. The aim of this paper is to describe what we learned about improving access to PHC for vulnerable groups across multiple sites through use of a common theory-based program logic model and a common evaluation approach. This was the IMPACT initiative. METHODS: IMPACT's evaluation used a mixed methods design with longitudinal (pre and post) analysis of six interventions. The analysis for this paper included four of the six sites that met study criteria. These sites were located in Canada (Alberta, Quebec and Ontario) and Australia (New South Wales). Using the overarching logic model, unexpected findings were reviewed, and alternative explanations were considered to understand how the mechanisms of each intervention may have contributed to results. RESULTS: Each site addressed their local access problem with different strategies and from different starting points. All sites observed changes in patient abilities to access PHC and provider access capabilities. The combination of intended and observed consequences for consumers and providers was different at each site, but all sites achieved change in both consumer ability and provider capability, even in interventions where there was no activity targeting provider behaviors. DISCUSSION: The model helped to identify, explore and synthesize intended and unintended consequences of four interventions that appeared to have more differences than similarities. Similar outcomes for different interventions and multiple impacts of each intervention on abilities were observed, implying complex causal pathways. CONCLUSIONS: All the interventions were a low-cost incremental attempt to address unmet health care needs of vulnerable populations. Change is possible; sustaining change may be more challenging. Access to PHC requires attention to both patient abilities and provider characteristics. The logic model proved to be a valuable heuristic tool for defining the objectives of the interventions, evaluating their impacts, and learning from the comparison of 'cases'.

Assessing primary care organization and performance: Literature synthesis and proposition of a consolidated framework.

BACKGROUND AND OBJECTIVES: Many frameworks describing primary care (PC) organization exist. This study proposes a consolidated framework based on the synthesis of published frameworks for the assessment of primary care organization and performance. APPROACH: We conducted a review of the literature to identify relevant existing frameworks that aimed to describe PC organization or/and monitor its activities. First, we extracted all domains from the frameworks and then hierarchically organized them into domains, dimensions and elements. Second, we mapped key domains. Third, we grouped together domains covering the same field to build a consolidated framework. Finally, the consolidated framework was assessed by 10 international experts in PC evaluation using a survey. RESULTS: We retained seven frameworks. The consolidated framework comprises four domains: 1) population needs; 2) organization and structure of PC practices; 3) delivery of PC services and 4) patient and population health outcomes. We added five connecting constructs to the framework in order to link the domains: accessibility, appropriateness, productivity, efficiency, effectiveness, equity and integration. None of the previously published frameworks encompassed all domains, dimensions and elements of the new consolidated framework. CONCLUSION: We propose a consolidated framework of PC organization based on the synthesis of seven published frameworks. This unitary framework may provide a foundation for comparative assessment across various contexts to support researchers and policy makers.

Implementing large-system, value-based healthcare initiatives: a realist study protocol for seven natural experiments.

INTRODUCTION: Value-based healthcare delivery models have emerged to address the unprecedented pressure on long-term health system performance and sustainability and to respond to the changing needs and expectations of patients. Implementing and scaling the benefits from these care delivery models to achieve large-system transformation are challenging and require consideration of complexity and context. Realist studies enable researchers to explore factors beyond 'what works' towards more nuanced understanding of 'what tends to work for whom under which circumstances'. This research proposes a realist study of the implementation approach for seven large-system, value-based healthcare initiatives in New South Wales, Australia, to elucidate how different implementation strategies and processes stimulate the uptake, adoption, fidelity and adherence of initiatives to achieve sustainable impacts across a variety of contexts. METHODS AND ANALYSIS: This exploratory, sequential, mixed methods realist study followed RAMESES II (Realist And Meta-narrative Evidence Syntheses: Evolving Standards) reporting standards for realist studies. Stage 1 will formulate initial programme theories from review of existing literature, analysis of programme documents and qualitative interviews with programme designers, implementation support staff and evaluators. Stage 2 envisages testing and refining these hypothesised programme theories through qualitative interviews with local hospital network staff running initiatives, and analyses of quantitative data from the programme evaluation, hospital administrative systems and an implementation outcome survey. Stage 3 proposes to produce generalisable middle-range theories by synthesising data from context-mechanism-outcome configurations across initiatives. Qualitative data will be analysed retroductively and quantitative data will be analysed to identify relationships between the implementation strategies and processes, and implementation and programme outcomes. Mixed methods triangulation will be performed. ETHICS AND DISSEMINATION: Ethical approval has been granted by Macquarie University (Project ID 23816) and Hunter New England (Project ID 2020/ETH02186) Human Research Ethics Committees. The findings will be published in peer-reviewed journals. Results will be fed back to partner organisations and roundtable discussions with other health jurisdictions will be held, to share learnings.